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BY  THE  U.S.  GENERAL  ACCOUNTING  OFFICE” 

Report  To  The  Secretary  Of  The  Navy 


AD-A155  036 


Acquisition  Of  Navy  Land-Based  Test 
Sites  Can  Be  Better  Managed 


This  report  discusses  ways  that  the  Navy 
can  improve  the  acquisition  management 
of  land-based  test  sites  to  ensure  that 
maximum  use  is  made  of  existing  facilities, 
duplication  between  existing  and  new  sites 
is  prevented,  and  the  establishment  of  new 
sites  is  cost-effective. 

GAO  makes  several  recommendations  de¬ 
signed  to  improve  the  acquisition  manage¬ 
ment  land-based  test  sites.  The  Oepart- 
ment.of  Defense  generally  agreed 
GAO's  findings  and  recommendations  and 
outlined  actions  planned  to  implement 
these  recommendations. 
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UNITED  STATES  GENERAL  ACCOUNTING  OFFICE 
WASHINGTON.  D.C.  20548 


NATIONAL  SECURITY  AND 
INTERNATIONAL  AFFAIR*  DIVISION 

B-217964 


The  Honorable  John  F.  Lehman 
The  Secretary  of  the  Navy 

Dear  Mr.  Secretary: 

Improvements  are  needed  in  the  acquisition  management  of 
land-based  test  sites  to  ensure  that  maximum  use  is  made  of 
existing  facilities,  duplication  between  existing  and  new  sites 
is  prevented,  and  the  establishment  of  new  sites  is  cost- 
effective  . 


The  Navy  has  numerous  physical  resources  for  providing 
research,  development,  test,  evaluation,  training,  and  opera¬ 
tional  support  for  weapons  systems  and  is  acquiring  more.  Land- 
based  test  sites  are  part  of  these  resources.  They  are  acquired 
to  perform  a  broad  range  of  tasks  including  test  and  evaluation, 
integration  of  components,  training,  and  operational  support. 

The  Navy  does  not  have  data  on  the  total  number  or  the  value  of 
these  sites.  However,  for  those  land-based  test  sites  having 
cost  data  available,  facilities  and  equipment  ranged  in  value 
from  $5  million  to  over  $400  million  a  site. 


The  acquisition  of  land-based  test  sites  is  governed  by 
Naval  Material  Command  Instruction  3960.8.  This  instruction  re¬ 
quires  that  maximum  use  be  made  of  existing  resources  before 
establishing  new  test  sites.  To  support  this  requirement,  the 
instruction  further  requires  that  (1)  systems  commands  (e.g., 
air,  sea,  electronic)  maintain  inventories  of  existing  facilities 
and  (2)  program  managers  cite  their  rationales  for  site  selection 
in  program  planning  documents. 


Land-based  test  sites  can  perform  many  services  in  the 
weapons  development  and  acquisition  process,  but  the  Navy  often 
does  not  adequately  demonstrate  the  need  for  new  facilities  prior 
to  purchase.  We  found: 


— Some  Navy  officials  were  not  aware  of  the  Naval 
Material  Command  instruction  and  others  did  not 
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think  the  instruction  applied  to  their  proqrams. 

As  a  result,  the  instruction  requirements  for 
justifying,  reviewing,  and  approving  the  acquisi¬ 
tion  of  new  land-based  test  sites  were  not  beinq 
followed . 

— Neither  the  Navy  nor  the  systems  commands  main¬ 
tained  complete  and  up-to-date  inventories  of 
available  test  sites.  Without  complete  inven¬ 
tories,  new  sites  may  be  established  that  duplicate 
existing  facilities  because  requesting  and  review¬ 
ing  officials  are  not  aware  of  them. 

— Most  of  the  acquisition  proqrams  had  no  documenta¬ 
tion  to  show  how  the  need  for  a  site  was  identi¬ 
fied,  whether  alternatives  were  considered,  and  why 
the  particular  alternative  was  chosen. 

— The  Naval  Material  Command  instruction  does  not 
make  cost-effectiveness  a  requisite  for  establish¬ 
ing  a  new  land-based  test  site.  Chief  of  Naval 
Operations  and  Naval  Material  Command  officials 
said  they  rely  on  the  inteqrity  and  knowledge  of 
the  program  manager  to  properly  evaluate  the  need 
for  the  site  and  they  do  not  require  that  the 
manager  demonstrate  cost-effectiveness  before 
approving  the  site. 

— The  site  review  and  approval  orocess  does  not  as¬ 
sure  that  a  proposed  site  is  needed.  The  lack  of 
site  inventories  and  other  evaluation  techniques 
places  too  much  reliance  on  the  personal  knowledge 
of  the  reviewers. 

Therefore,  the  Navy  does  not  have  as  much  assurance  as  it 
could  that  the  new  sites  it  purchases  are  the  most  cost-effective 
alternatives  and  that  they  do  not  duplicate  existinq  facilities. 
For  example,  the  Naval  Sea  Systems  Command  permitted  the  Aeqis 
program  office  to  establish  a  contractor  land-based  test  site 
to  integrate  and  test  a  revised  antisubmarine  warfare  combat 
system  for  the  Aegis  cruiser  and  DDG-51  destroyer.  The  Naval 
Material  Command’s  Naval  Ocean  Systems  Center  had  integrated  and 
tested  earlier  versions  of  the  combat  system,  had  the  Navy  exper¬ 
tise  and  facilities  to  do  the  same  functions  for  the  revised 
system,  and  believed  it  could  integrate  and  test  the  revised 
system  at  less  cost  than  the  new  site  could.  Any  doubt  as  to 
whether  establishment  of  the  new  site  was  in  the  best  interest  of 
the  Navy  could  have  been  resolved  had  decisionmakers  considered 
all  pertinent  data  and  made  appropriate  cost-benefit  analyses. 

Navy  officials  agreed  that  problems  exist  and  have  begun  to 
make  changes  in  regulations  and  recordkeeping  activities,  but 
actions  are  not  yet  complete.  While  these  actions  are  steps  in 
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the  right  direction,  we  believe  more  can  be  done.  Therefore,  we 
recommend  you  direct  that: 

— The  Naval  Material  Command  instruction  clearly 
state  (1)  which  programs  and  sites  are  subject  to 
its  requirements  and  (2)  that  the  sites  selected  be 
evaluated  on  the  basis  of  cost-effectiveness  and 
needed  capability. 

— The  Chief  of  Naval  Operations  or  the  Naval  Material 
Command  instruction  be  revised  so  that  the  instruc¬ 
tions  agree  on  the  responsibility  for  identifying 
the  need  for  land-based  test  sites.  Currently,  the 
Naval  Material  Command  instruction  assigns  this 
responsibility  to  the  program  manager  whereas  the 
Naval  Operations  instruction  makes  the  Chief 
responsible . 

— The  Naval  Material  Command  requirement  that 

systems  commands  maintain  complete  and  up-to-date 
inventories  of  in-house  and  contractor  test  sites 
be  enforced  to  aid  in  making  acquisition  decisions, 
and  the  Chief  of  Naval  Operations  establish  a 
central  Navy-wide  inventory  of  test  and  evaluation 
facilities. 

— Program  managers  consider  alternatives  in  selecting 
sites  and  include  their  rationale  in  the  planning 
documents. 

— The  Chief  of  Naval  Operations  and  Naval  Material 
Command  program  review  and  approval  process  include 
evaluation  techniques  that  consider  the  cost- 
effectiveness  of  sites  selected,  the  availability 
of  existing  facilities,  and  the  extent  of  dup¬ 
lication  between  existing  and  proposed  sites. 

In  providing  official  written  comments  on  a  draft  of  this 
report,  the  Department  of  Defense  generally  agreed  with  our 
findings  and  recommendations.  Our  findings,  recommendations,  and 
agency  comments  are  discussed  in  more  detail  in  appendix  I.  The 
Department's  written  comments  are  included  as  appendix  II. 

As  you  know,  31  U.S.C.  S720  requires  the  head  of  a  federal 
agency  to  submit  a  written  statement  on  actions  taken  on  our  rec¬ 
ommendations  to  the  Senate  Committee  on  Governmental  Affairs  and 
the  House  Committee  on  Government  Operations  not  later  than  60 
days  after  the  date  of  the  report  and  to  the  House  and  Senate 
Committees  on  Appropriations  with  the  agency's  first  request  for 
appropriations  made  more  than  60  days  after  the  date  of  the 
report. 
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We  are  sending  copies  of  this  report  to  the  Chairmen  of  the 
above  committees;  the  Chairmen,  Senate  and  House  Committees  on 
Armed  Services;  the  Secretary  of  Defense;  and  the  Director, 
Office  of  Management  and  Budget. 


Sincerely  yours, 


<2 


Frank  C.  Conahan 
Director 
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ACQUISITION  OF  NAVY  LAND-BASED 
TEST  SITES  CAN  BE  BETTER  MANAGED 


The  Navy  operates  a  variety  of  facilities  with  the  capabili¬ 
ties  to  do  research,  development,  and  testing  for  weapons  systems 
and  their  components  and  is  acquiring  more.  These  facilities 
range  from  small  research  and  development  laboratories  to  opera¬ 
tional  test  ranges  encompassing  thousands  of  sauare  miles.  The 
facilities  have  highlv  sophisticated  computers  and  other  equip¬ 
ment  to  evaluate  every  phase  of  a  weapons  system  from  basic  re¬ 
search  to  actual  wartime  scenarios.  As  weapons  systems  have 
become  more  complex,  the  need  for  test  and  evaluation  has  in¬ 
creased  . 

In  the  mid-1960s,  the  Navy  adopted  land-based  testinq  to 
reduce  development,  construction,  and  repair  problems.  This 
testinq  was  expanded  in  the  early  1970s  to  help  ensure  new 
weapons  systems  and  their  components  were  viable  before  acauisi- 
tion  and  installation.  Land-based  test  sites  are  facilities, 
owned  either  by  the  Navy  or  by  its  contractors,  that  test,  re¬ 
fine,  and  integrate  weapons  systems  being  purchased  or  undergoing 
changes.  The  sites  are  usually  dedicated  to  a  specific  weapon 
system,  such  as  a  ship,  aircraft,  or  missile,  or  to  one  of  its 
major  subsystems,  such  as  the  combat  subsystem  or  the  prooulsion 
subsystem. 

Systems  that  have  been  tested  and  shown  to  work  before  being 
purchased  or  altered  are  less  likely  to  cause  installation  and 
operational  problems.  Navy  officials  cited  the  following 
benefits  as  resulting  from  land-based  test  sites. 

— Reduction  in  the  amount  of  defective  materials 
provided  by  the  government  to  contractors,  with  a 
consequent  reduction  in  the  number  of  contractor 
claims. 

— Shorter  construction  periods. 

— Reduction  in  time  between  government  acceptance  of 
a  major  system  and  achievement  of  fully  operation¬ 
al,  deployable  status. 

— Development  of  internal  Navy  expertise  for  identi¬ 
fying  and  evaluating  system  or  equipment  problems, 
chanqes,  and  contractor  proposals. 

— Timely,  effective  Life-cycle  support  that  elimi¬ 
nates  the  need  to  remove  an  active  system  from 
duty  in  order  to  evaluate  a  problem. 
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Orqanizat ional ly ,  the  weapon  system  proqram  manaqer  is 
responsible  for  identifying  the  need  for  a  land-based  test  site. 
The  responsibility  for  reviewinq  and  approving  a  site  extends 
upward  through  the  Navy  organizat ional  structure,  from  the 
systems  command — Naval  Air  Systems  Command  (NAVAIR),  Naval  Sea 
Systems  Command  (NAVSEA),  or  Naval  Electronic  Systems  Command 
(NAVSLEX) — to  the  Naval  Material  Command  (NAVMAT )  and  to  the 
Office  of  the  Chief  of  Naval  Operations  (CNO). 

In  the  two  decades  since  the  Navy  located  its  first  site  at 
the  Philadelphia  Naval  Shipyard,  the  number  of  sites  and  func¬ 
tions  have  increased,  but  the  Navy  was  unable  to  tell  us  the 
total  number  of  sites  it  and  contractors  currentlv  have  or  are 
acquiring.  Today,  NAVAIR,  NAVSEA,  and  NAVELEX  have  land-based 
test  sites.  Of  the  12  Navy  and  contractor  land-based  test  sites 
we  reviewed,  facility  and  equipment  costs  ranged  from  an 
estimated  $5  million  for  a  HARM  missile  software  support  activity 
to  over  $400  million  for  an  Aegis  cruiser  combat  systems 
engineering  development  site. 

The  NAVMAT  instruction  (3960.8)  governing  land-based  test 
sites  permits  a  broad  ranqe  of  functions  to  be  performed, 
including : 

— Developmental  or  operational  test  and  evaluation. 

— Integration  and  interface  testinq  of  equipment  and 
subsystems. 

— Verification  of  subsystem  equipment. 

— Development  or  verification  of  technical  documen¬ 
tation  and  production  acceptance  test  procedures. 

— Verification  and  validation  of  operational  and 
maintenance  computer  software. 

— Initial  operational  or  maintenance  traininq. 

— Proof  testing  alterations  or  modifications. 

— Verification  of  operator  and  crew  task  assign¬ 
ments. 

— Verification  of  installation  and  check-out  proce¬ 
dures. 

— Preinstallation  testing  and  check-out  of  produc¬ 
tion  hardware. 

A  recent  draft  revision  of  the  instruction  also  designates 
computer  software  support  activities  as  land-based  test  sites. 
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Most  of  these  functions  are  also  performed  by  other  Navy 
facilities  owned  by  the  systems  commands  and  NAVMAT.  However, 
the  Navy  does  not  consider  these  other  sites  to  be  land-based 
test  sites.  Land-based  test  sites  generally  support  individual 
weapon  acquisition  programs  and  are  designed  to  satisfy  specific 
program  needs.  The  other  sites  perform  similar  functions  but 
tend  to  be  more  general  purpose  or  functionally  oriented  than 
land-based  test  sites. 

OBJECTIVE,  SCOPE, 

AND  METHODOLOGY 


Given  the  significant  investment  represented  by  land-based 
test  sites  and  their  growing  numbers  in  the  Navy,  our  review 
sought  to  determine  how  well  the  Navy  ensures  that  acquisition  of 
new  sites  is  cost-effective  and  does  not  duplicate  existing 
facilities.  In  pursuing  this  objective,  we  examined  (1)  the 
controls  for  justifying  and  establishing  sites  and  (2)  the 
adequacy  of  the  review  and  approval  process. 

We  traced  the  acquisition  and  use  of  land-based  test  sites 
and  identified  facilities  doing  similar  functions  through  exami¬ 
nation  of  Navy  documents  and  discussions  with  Navy  officials. 

For  10  of  the  12  land-based  test  sites  included  in  our  review,  we 
followed  the  process  from  the  initial  identification  of  need  for 
the  sites  through  installation  and  use  of  weapon  systems  that  had 
been  tested  at  the  sites.  We  attended  briefings,  toured 
facilities,  and  analyzed  documents.  Also,  we  interviewed 

— program  managers  and  staff  from  NAVAIR,  NAVSEA, 
and  NAVE LEX; 

— reviewing  officials  from  NAVMAT  and  CNO; 

— Navy  and  contractor  officials  at  land-based  test 
sites; 

— Navy  and  contractor  officials  responsible  for 
installing  and  testing  weapons  systems;  and 

— Navy  officials  responsible  for  monitoring  and 
providing  in-service  support  to  the  fleet. 

Two  limitations  affected  the  scope  of  this  review.  First, 
we  were  unable  to  evaluate  the  cost-effectiveness  of  the  land- 
based  test  sites.  The  Navy  does  not  maintain  quantifiable  data 
on  test  site  efficiency,  effectiveness,  or  economy,  and  we  could 
not  obtain  reliable  data  with  which  to  make  an  independent  evalu¬ 
ation.  Second,  for  most  of  the  sites,  we  could  not  determine 
whether  program  requirements  had  been  compared  with  available 
Navy  test  site  resources  before  new  sites  were  acquired  or  if  the 
best  alternative  wa3  selected.  Program  managers  usually  did  not 
maintain  documentation  which  would  provide  answers  to  these  ques¬ 
tions. 
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We  performed  oar  review  from  January  1984  to  December  1984 
at  the  following  locations: 

Headquarters  Commands 

Office  of  the  Chief  of  Naval  Operations,  Washington,  D.C. 
Naval  Material  Command,  Washington,  D.C. 

Naval  Air  Systems  Command,  Washington,  D.C. 

Naval  Electronic  Systems  Command,  Washington,  D.C. 

Naval  Sea  Systems  Command,  Washington,  D.C. 

Operational  Test  and  Evaluation  Force,  Norfolk,  Virginia 
Inspection  and  Survey  Board,  Washington,  D.C. 

Naval  Laboratories,  Test  Facilities,  Ranges 

Naval  Weapons  Center,  China  Lake,  California 
Pacific  Missile  Test  Center,  Point  Mugu,  California 
Naval  Ocean  Systems  Center,  San  Diego,  California 
Naval  Underwater  Systems  Center,  New  London,  Connecticut 

Land-Based  Test  Sites 


Aegis  Combat  Systems  Engineering  Design  Site,  Moorestown, 

New  Jersey 

Aegis  RCA  Production  Test  Center,  Moorestown,  New  Jersey 
FFG-7  Combat  Systems  Test  Center,  Ronkonkama,  New  York 
Integrated  Combat  Systems  Test  Facility,  San  Diego, 
California 

Electronic  Warfare  Systems  Support  Laboratory,  Pacific 
Missile  Test  Center,  Point  Mugu,  California 
F-14  Software  Support  Activity,  Pacific  Missile  Test  Center, 
Point  Mugu,  California 

F/A-18  Weapons  System  Support  Activity,  Naval  Weapons 
Center,  China  Lake,  California 
HARM  Missile  Software  Support  Activity,  Naval  Weapons 
Center,  China  Lake,  California 
Naval  Electronic  Systems  Command  FFG-7  Test  Site, 

Charleston,  South  Carolina 

DD-963  and  LHA-5  Combat  System  Test  Facility,  Ingalls 
Shipyard,  Pascagoula,  Mississippi 
U.S.S.  Norton  Sound,  Port  Hueneme,  California 

Other  Navy  Sites/Facilities 

Naval  Ship  Weapons  System  Engineering  Station,  Port  Hueneme, 
Cal ifornia 

Harpoon  Missile  Laboratory,  Naval  Weapons  Center,  China 
Lake,  California 

Harpoon  Engineering  Support  Office,  Pacific  Missile  Test 
Center,  Point  Mugu,  California 
Production  Acceptance  Test  and  Evaluation  Facility,  Pacific 
Missile  Test  Center,  Point  Mugu,  California 
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Range  Control  Center,  Naval  Weapons  Center,  China  Lake, 
California 

Range  Operations  Center,  Pacific  Missile  Test  Center,  Point 
Mugu,  California 

We  also  contacted  the  program  office  and  reviewed  documentation 
for  the  LAMPS  Mark  III  helicopter  program’s  land-based  test  site, 
but  we  did  not  visit  the  actual  test  site  in  New  York. 

Our  review  was  made  in  accordance  with  generally  accepted 
government  audit  standards. 

NAVMAT  INSTRUCTION 
NOT  ALWAYS  FOLLOWED 


NAVMAT  Instruction  3960.8  allows  a  wide  range  of  functions 
to  be  performed  by  land-based  test  sites.  The  same  types  of 
functions  are  performed  by  numerous  other  Navy  activities  but 
they  are  not  classified  as  land-based  test  sites.  Therefore, 

Navy  officials  are  confused  as  to  the  applicability  of  this 
instruction  to  their  programs.  Some  Navy  officials  were  not 
aware  of  the  NAVMAT  instruction.  Others  did  not  think  the  in¬ 
struction  applied  to  their  programs.  As  a  result,  the  instruc¬ 
tion  requirements  for  justifying,  reviewing,  and  approving  the 
acquisition  of  new  land-based  test  sites  were  not  followed. 

This  problem  is  illustrated  at  the  China  Lake  Naval  Weapons 
Center.  It  has  a  number  of  sites  that  provide  the  type  of 
support  authorized  for  land-based  test  sites.  The  Center  also 
supports  programs  in  the  acquisition  categories  listed  in  the 
instruction.  However,  Center  officials  said  they  were  not  aware 
of  the  instruction  prior  to  our  review  and  questioned  whether  it 
actually  applied  to  their  sites.  They  also  said  they  planned  to 
determine  its  applicability  to  their  programs  for  future  actions. 

NAVAIR  and  NAVSEA  officials  did  not  believe  that  the 
instruction  was  applicable  to  some  of  their  sites.  NAVAIR 
officials  did  not  regard  computer  software  support  activities  as 
land-based  test  sites,  although  most  headquarters  and  program 
officials  interviewed  agreed  the  sites  support  programs  and 
perform  functions  described  in  the  NAVMAT  instruction.  NAVSEA 
officials  thought  parts  of  the  instruction  were  no  longer  valid 
because  of  changes  made  in  August  1983  to  an  Operations  Navy 
( OPNAV )  instruction  referenced  by  the  NAVMAT  instruction.  This 
instruction  (OPNAV  Instruction  3960. 10B)  designates  the  Chief  of 
Naval  Operations  as  the  person  responsible  for  determining  the 
need  for  land-based  test  sites  for  ship  acquisition  programs 
whereas  the  NAVMAT  instruction  assigns  the  responsibility  to  the 
program  manager. 
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NAVMAT  officials  stated  that  they  would  revise  NAVMAT 
Instruction  3960.8  to  better  define  the  sites  subject  to  the  in¬ 
struction.  Software  support  activities  will  be  addressed  as 
one  of  the  possible  functions  of  a  land-based  test  site.  CNO 
officials  stated  that  the  change  to  the  OPNAV  instruction  was  not 
intended  to  relieve  program  managers  of  the  responsibility  for 
determining  the  need  for  land-based  test  sites  but  rather  to 
ensure  the  program  managers  understand  that  the  sites  must  be 
approved  by  CNO.  CNO  officials  also  stated  that  they  would  re¬ 
vise  OPNAV  Instruction  3960. 10B  to  make  it  clear  that  program 
managers  are  responsible  for  identifying  the  need  for  land-based 
test  sites. 

COMPLETE  INVENTORIES  OF  EXISTING 
SITES  NOT  MAINTAINED 

Neither  the  Navy  nor  the  systems  commands  maintained  a  cen¬ 
tral  or  a  complete  systems  command  inventory  of  available  test 
sites.  Without  complete  inventories,  new  sites  may  be  estab¬ 
lished  that  duplicate  existing  facilities  because  requesting  and 
reviewing  officials  may  not  be  aware  of  them. 

According  to  the  NAVMAT  instruction,  before  a  new  test  site 
is  acquired,  officials  must  compare  their  needs  to  the  assets  and 
capabilities  of  existing  test  sites.  This  comparison  should 
ensure  both  the  maximum  utilization  of  existing  sites  and  the 
timely,  cost-effective  acquisition  and  development  of  new  ones. 
The  instruction  assigns  the  responsibility  for  making  this 
comparison  to  the  program  manager. 

The  instruction  also  directs  each  Navy  systems  command  to 
establish  a  focal  point  to  provide  information  on  sites  and 
their  capabilities.  The  focal  point  is  to  maintain  a  site 
inventory  for  the  systems  command  and  to  coordinate  site  avail¬ 
ability  reviews  among  the  commands.  However,  the  instruction 
does  not  require  the  focal  points  or  any  other  office  to  maintain 
a  central  inventory  of  all  available  Navy  test  sites  and  their 
capabilities. 

The  systems  commands  maintained  only  partial  or  no 
inventories  of  test  sites.  NAVAIR  had  a  central  inventory  of 
software  support  activities,  but  it  did  not  include  other  NAVAIR 
Land-based  test  sites.  NAVSEA  had  a  partial  listing  of  contrac¬ 
tor  sites  and  a  more  complete  list  of  internal  Navy  sites. 

NAVELEX  had  no  inventory.  Without  complete  inventories,  the  Navy 
may  establish  new  sites  that  duplicate  existing  facilities. 

Also,  a  complete  facility  might  be  purchased  when  a  less 
expensive  modification  to  an  existing  facility  could  have  been 
made . 
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As  an  example  of  possible  overlap  or  duplication,  both 
NAVMAT’s  China  Lake  Naval  Weapons  Center  and  NAVAIR’s  Pacific 
Missile  Test  Center  provide  test  support  to  the  Harpoon  missile 
proqram.  In  addition,  NAVSEA’s  Naval  Ship  Weapons  System 
Engineering  Station  recently  completed  a  new  laboratory  that 
also  provides  engineering  test  services  for  the  Harpoon.  All 
three  sites  are  within  a  200-mile  radius,  yet  there  is  no  single 
inventory  that  shows  the  three  sites  and  distinguishes  among  the 
services  provided  by  each.  Officials  at  the  first  two  sites  did 
not  know  of  the  new  laboratory's  existence. 

CNO  officials  told  11s  that  their  Test  and  Evaluation 
Division  would  prepare  a  central  inventory  of  test  sites  for  the 
Navy.  NAVSEA  officials  stated  that  they  were  developing  a  better 
inventory  of  land-based  test  sites,  as  well  as  sea-based,  for 
their  command. 

In  commenting  on  our  draft  report,  the  Department  of  Defense 
(DOD)  agreed  that  a  central  inventory  had  merit  and  stated  that 
one  would  be  developed.  DOD  stated,  however,  that  a  land-based 
test  site  of  one  command  would  be  of  little  use  to  another 
command.  Althouqh  this  statement  may  be  generally  correct,  we 
believe  that  in  some  cases  a  test  site  could  have  cross-command 
applicability.  For  example,  the  LAMPS  Mark  TIT  land-based  test 
site  is  a  ioint  NAVAIR/NAVSEA  site. 

DOD  further  stated  that  the  three  test  sites  in  the  Harpoon 
example  are  described  in  NAVSEA' s  test  and  range  facilities 
catalog.  We  reviewed  the  catalog  and  found  that  the  facilities 
were  described  in  general  terms,  but  the  test  support  provided 
the  Harpoon  program  was  mentioned  for  only  one  of  the  sites. 

SITE  SELECTION  RATIONALE 
SELDOM  DOCUMENTED 

Most  of  the  acquisition  programs  we  reviewed  had  no 
cumentation  to  show  how  the  need  for  a  site  was  identified, 
•ther  alternatives  were  considered,  and  why  the  particular 
a  ernative  was  chosen.  Because  CNO  and  NAVMAT  reviewers  rely  on 
t  program  manager  to  properly  evaluate  the  need  for  a 
ind-based  test  site  and  the  available  alternatives,  we  reviewed 
v  le  manager's  decisionmakinq  process. 

Acquisition  program  officials  stated  that  thev  base  their 
site  research,  cost  comparisons,  and  other  analyses  upon  the 
personal  knowledge  or  expertise  of  their  staffs.  In  the  absence 
of  a  complete  list  of  available  Navy  facilities,  they  depend  upon 
their  staff  members'  personal  knowledge  to  ensure  that  test  sites 
do  not  overlap  or  duplicate  other  Navy  activities  or  sites. 


1 1 


APPENDIX 


APPENDIX  I! 


I  r 


RECOMMENDATIONS 

•  RECOMMENDATION  I:  GAO  recommended  the  Secretary  of  the  Nav 
direct  that  the  NAVMAT  instruction  clearly  state  (1)  which 
programs  and  sites  are  subject  to  its  requirements  and  (2)  that 
the  sites  selected  be  evaluated  on  the  basis  of  cost- 
effectiveness  and  needed  capability.  [See  pp.  3  and  15.  ( 

POD  Response.  Concur.  NAVMAT  Instruction  3960.8  will  re 
revised  by  15  August  1985.  Cost-effectiveness  will  be 
addressed  as  stated  in  our  response  to  "Finding  F." 

•  RECOMMENDATION  2 ;  GAO  recommended  the  Secretary  of  t..‘  Nav 
direct  that  tho  OPNAY  or  NAVMAT  instructions  be  revised  so 
that  they  agree  on  the  responsibilities  for  identifying  ths 
need  for  land-based  test  sites.  [See  on.  3  and  15.  : 

POD  Response.  Concur.  OPNAY  Instruction  3960. 10B  and  NAV". 
Instruction  3960.8  will  be  revised  by  15  August  1985. 

•  RECOMMENDATION  3 :  GAO  recommended  the  Secretary  of  the  Nav 
direct  that  the  NAVMAT  requirement  that  Systems  Commands 
maintain  complete  and  up-to-date  inventories  of  in-house  an 
contractor  test  sites  be  enforced  to  aid  in  making 
acquisition  decisions;  and  that  the  Chief  of  Naval  Operatic 
establish  a  central  Navy-wide  inventory  of  test  and 
evaluation  facilities.  [See  pp.  3  and  15.) 

POD  Response.  Concur.  NAVMAT  Instruction  3960.8  will  be 
revised  by  15  August  1985.  NAVMAT  will  establish  a  central 
inventory  for  CNO  by  15  December  1985. 

•  RECOMMENDATION  4 ;  GAO  recommended  the  Secretary  of  the  Nav 
direct  that  program  managers  consider  alternatives  in 
selecting  sites  and  include  their  rationales  in  the  plarnin 
documents.  [See  pp .  3  and  15.  | 

POD  Response.  Concur.  NAVMAT  Instruction  3960.8  will  be 
revised  by  15  August  1985. 

•  RECOMMENDATION  5 :  GAO  recommended  the  Secretary  of  the  Nav 
direct  that  the  Chief  of  Naval  Operations  and  NAVMAT  progra 
review  and  approval  process  include  evaluation  techniques 
which  consider  the  cost-effectiveness  of  the  site  selected, 
the  availability  of  existing  facilities,  and  the  extent  of 
duplication  between  existing  and  proposed  sites.  [See 

pp .  3  and  15  .  | 

POD  Response.  Concur.  NAVMAT  Instruction  3960.8  will  be 
revised  by  15  August  1935  to  clarify  and  provide  guidance  o 
the  elements  of  the  review  and  approval  process. 
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Site  (ASEDS) .  The  ASWPTS  and  ASEDS  were  established  to 
support  the  installation  and  testing  of  the  Surface  Ship  ASW 
Combat  Systems  being  installed  aboard  DDG  51  and  CG  47  Class 
ships.  These  sites  do  not  duplicate  any  existing  land-based 
sites,  including  those  indentified  on  page  9  of  the  report. 

In  addition,  the  functions  being  performed  at  these  two  sites 
are  significantly  different  than  those  being  performed  at  any 
Navy  activity,  including  NOSC.  Specifically,  the  ASWPTS  was 
established  to  test,  in  an  integrated  fashion,  production  ASW 
subsystems  as  an  ASW  combat  system,  conduct  an  end-to-end 
test  from  the  sensors  through  the  fire  control,  and  to  serve 
as  a  staging  area  for  the  shipment  of  the  fully  tested  and 
integrated  ASW  Combat  System  to  the  shipyard  for  installation 
in  AEGIS  cruisers  and  destroyers.  The  purpose  of  ASEDS  is  to 
develop  and  test  engineering  changes  for  the  ASW  Combat 
System  being  installed  aboard  DDG  51  class  of  AEGIS 
destroyers.  It  consists  of  in-house  and  configuration 
control  model  equipment,  most  of  which  was  previously  located 
at  Syracuse,  interconnected  to  form  an  ASW  Combat  System 
similar  to  that  being  installed  aboard  DDG  51.  Costs 
associated  with  these  two  sites  are  significantly  less  than 
the  $50  million  identified.  Costs  associated  with 
establishment  of  the  ASWPTS  and  the  ASEDS  combined  are  less 
than  $15.0  million.  The  decision  to  use  the  Syracuse  sites 
resulted  from  an  extensive  study  conducted  by  NAVSEA  in  1982. 
Sites  considered  included  NOSC  as  well  as  many  others.  The 
Syracuse  site  was  determined  to  be  the  only  one  where  the 
required  activities  could  be  accomplished  in  a  cost  effective 
and  timely  manner.  OPNAV  did  approve  it.  Site  selection 
documentation  is  and  has  been  available  in  NAVSEA. 
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establish  the  new  $50  million  site.  In  addition,  since 
AEGIS  officials  did  not  consider  this  a  land-based  site, 
it  did  not  follow  the  NAVMAT  instruction. 

GAO  concluded  that  the  land-based  test  site  process  should 
include  an  evaluation  of  cost-effectiveness.  [See  po.  2,  12, 
13  ,  and  14 .  ] 

POD  Response .  Partially  concur.  DOD  disagrees  that 
NAVMAT IN ST  3960.8  does  not  make  cost  effectiveness  a 
requisite  for  establishing  a  LBTS.  The  following  excerpts 
from  that  instruction  pertain: 

a.  "Fundamental  to  the  selection  site  location  is  the  need 
to  obtain  maximum  benefit  to  the  Navy  for  resources 
invested . " 

b.  An  objective  of  the  instruction  is  "...  to  ensure  the 
comparison  of  all  projects  planned  LBTS  requirements  vs. 
inventory  and  assets,  thereby  ensuring  the  maximum 
utilization  of  existing  and/or  timely  and  cost  effective 
development/acquisition  of  a  new  LBTS." 

c.  Two  factors  that  bear  on  decisions  to  establish  a  LBTS 
are  (1)  "maximum  use  of  existing  facilities  ...  should  be 
pursued  when  ...  cost  benefits  favorably  indicate  this  as  a 
rational  approach"  and  (2)  "an  examination  of  the  elements  of 
life  cycle  costs  relative  to  alternative  site  locations  ... 
must  be  made." 

DOD  agrees  that  cost  effectiveness  should  be  more 
formally  addressed  in  our  documentation.  However,  the 
specific  technical  problems  cannot  be  foreseen  in  a  given 
program  and  the  cost  of  correcting  them  at  a  LBTS  compared  to 
the  costs  of  correcting  them  later  cannot  be  meaningfully 
est imated . 

The  Department  also  believes  that  the  example  used  to 
support  Finding  F  is  unclear  and  possibly  inaccurate.  It  is 
not  clear  which  test  site  is  being  discussed  on  page  15  of 
the  report.  If  the  test  site  referred  to  is  one  of  the  two 
test  sites  at  General  Electric  in  Syracuse,  New  York,  the 
report  is  inaccurate.  These  are:  (1)  ASW  Production  Test 
Site  (ASWPTS) ;  and  (2)  ASW  System  Engineering  Development 
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how  the  need  for  the  site  was  identified,  whether 
alternatives  were  considered,  and  why  the  particular 
alternative  was  chosen.  GAO  reported  that,  according  to  the 
Navy,  program  acquisition  officials  base  their  site  research, 
cost  comparisons,  and  other  analyses  upon  the  personal 
knowledge  or  expertise  of  their  staffs.  GAO  concluded  that 
(instead  of  the  required  documentation)  the  land-based  test 
site  acquisition  is  based  primarily  on  the  personal  knowledge 
of  the  reviewers.  (GAO  noted  NAVMAT  advised  that  in  the 
future,  better  documentation  for  proposed  sites  would  be 
required.)  [See  pp.  1,  2,  11,  12,  and  14.] 

POD  Response.  Partially  concur.  The  DOD  disagrees  that  LBTS 
did  not  have  programming  documentation  that  supports  the  site 
selection  process.  All  LBTS  had  such  approval  documentation 
in  order  to  obtain  the  required  funding  but  it  was  not 
retained.  DOD  agrees  that  it  would  be  useful  to  retain 
records  to  document  how  the  need  for  a  particular  site  was 
identified,  what  alternatives  were  considered,  and  why  the 
final  alternative  was  chosen.  However,  the  draft  GAO  report 
states  that  according  to  the  Navy,  program  managers  base 
their  site  research,  cost  comparisons,  and  other  analyses  on 
the  personal  knowledge  or  expertise  of  their  staff.  The  GAO 
description  of  this  process  did  not  acknowledge  the  data  that 
does  exist  from  past  programs  and  is  used,  such  as  site 
management  plans  and  test  program  reports.  Nor  does  it 
recognize  the  staff  offices  located  in  the  Systems  Commands 
which  oversee  the  various  test  and  range  facilities  and  act 
as  liaisons  with  them. 

FINDING  F:  Cost  Effectiveness  Not  Required  By  NAVMAT 

Instruction .  GAO  found  that  NAVMAT  Instruction  3960.8  does 
not  make  cost-effectiveness  a  requisite  for  establishing  a 
new  land-based  test  site.  GAO  further  found  that  the  CNO  and 
NAVMAT  do  not  require  the  program  manager  to  demonstrate 
cost-effectiveness  before  approving  a  site. 

--  Example :  GAO  cited  the  Navy's  decision  to  establish  a 
contractor  site  to  integrate  and  test  a  revised  anti¬ 
submarine  warfare  combat  system  for  the  DDG-51  destroyer 
and  AEGIS  cruiser.  GAO  identified  NAVMAT's  Naval  Ocean 
Systems  Center  which  could  possibly  have  performed  the 
necessary  program  support  because  it  already  was 
integrating  and  testing  other  versions  of  this  combat 
system.  The  AEGIS  Program  Manager  chose  to  contract  on  a 
sole-source  procurement  basis  because  the  Government  did 
not  have  the  capability  to  perform  the  work  in-house.  The 
Center  was  not  contacted  prior  to  the  decision  to 
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officials  must  compare  their  needs  to  the  assets  and 
capabilities  of  existing  sites.  The  instruction  requires 
each  Systems  Command  to  maintain  a  site  inventory  and 
coordinate  site  availability  reviews  among  commands.  (It 
does  not,  however,  require  that  a  central  inventory  of  all 
available  sites  be  maintained,  and  their  capabilities.)  GAO 
found  that  the  Systems  Commands  do  not  maintain  an  up-to-date 
inventory  of  available  test  sites,  nor  does  the  Navy  maintain 
a  central  inventory. 

—  Example :  GAO  cited  the  Harpoon  missile  program  as  an  area 
where  possible  overlap  or  duplication  has  occurred.  Both 
NAVMAT's  China  Lake  Naval  Weapons  Center  and  NAVAIR's 
Missile  Test  Center  provide  test  support  to  the  Harpoon 
missile  program.  In  addition,  NAVSEA's  Naval  Ship  Weapons 
System  Engineering  Station  recently  completed  a  new 
laboratory  which  also  provides  engineering  test  services 
for  the  Harpoon.  All  three  sites  are  within  a  200-mile 
radius,  yet  there  is  no  single  inventory  which  shows  the 
three  sites  and  distinguishes  among  the  services  provided 
by  each.  (GAO  reported  that  officials  at  the  first  two 
sites  did  not  know  the  new  laboratory  existed.) 

GAO  concluded  that  without  complete  inventories,  the 
required  comparison  cannot  be  done,  and  therefore,  the 
Navy  may  establish  new  sites  that  overlap  or  duplicate 
existing  facilities.  GAO  also  concluded  that  a  complete 
facility  might  be  purchased  when  a  less  expensive 
modification  could  have  been  made  to  an  existing  facility. 
(GAO  noted  that  the  Navy  advised  its  Test  and  Evaluation 
Division  would  prepare  a  central  inventory  of  test  sites.) 
[See  pp.  2,  10,  11,  and  14.] 

QQP _ Response .  Concur.  However,  none  of  the  sites  addressed 

in  the  GAO  example  are  LBTS.  Nevertheless,  the  three  general 
purpose  test  sites  are  described  in  NAVSEA’s  "Test  and  Range 
Facilities  Catalog  •  Since  each  LBTS  is  unique  to  a  weapon 
system,  overlap  or  duplication  is  very  unlikely.  The  Navy 
has  only  about  twenty-two  LBTS.  LBTS  of  NAVSEA  would  be  of 
little  use  to  NAVELEX  or  NAVAIR  and  vice  versa. 
Notwithstanding  this,  DOD  agrees  a  central  inventory  has 
merit  and  it  will  be  developed. 

FINDING  E :  Site  Selection  Rationale  Seldom  Documented.  GAO 
reported  that  the  NAVMAT  instruction  requires  the  program 
managers  to  cite  their  rationales  for  site  selection  in 
programming  documents.  GAO  found,  however,  that  most  of  the 
acquisition  programs  it  reviewed  had  no  documentation  to  show 
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—  Example  2:  NAVAIR  officials  did  not  believe  the 

instruction  applied  to  some  of  their  sites  and  did  not 
regard  computer  software  support  activities  as  land-based 
test  sites,  although  some  agreed  the  sites  supported 
functions  illustrated  in  the  NAVMAT  instruction. 

—  Example  3:  NAVSEA  did  not  believe  the  NAVMAT  instruction 
was  valid  because  of  changes  made  in  August  1983  to  OPNAV 
Tr.«*-cuc*-  ’ 3960. 10B,  desiqnatinq  the  Chief  of  Naval 
Operations  as  the  responsible  entity  for  determining  the 
need  for  land-based  ship  acquisition  test  sites. 

GAO  concluded  that  because  the  same  types  of  functions  are 
performed  by  numerous  other  activities  which  are  not 
classified  as  land-based  test  sites,  some  Navy  officials 
are  confused  as  to  the  applicability  of  the  NAVMAT 
instruction  to  their  facilities.  In  addition,  GAO 
concluded  some  Navy  officials  are  simply  not  aware  of  the 
NAVMAT  instruction.  GAO  finally  concluded  that  management 
of  the  Navy's  land-based  test  site  acquisition  process  is 
not  working  as  effectively  as  it  could.  (GAO  noted  that 
NAVMAT  officials  had  stated  the  instruction  would  be 
revised  (1)  to  better  define  the  sites  subject  to  the 
instruction  and  (2)  to  include  software  support 
activities.  GAO  also  reported  Navy  officials  stated  the 
OPNAV  Instruction  3960. 10B  would  be  revised  to  make  it 
consistent  with  NAVMAT  instruction — i.e.,  that  program 
managers  are  responsible  for  determining  the  need  for 
land-based  test  site  and  the  CNO  is  responsible  for 
approving  the  acquisition.)  [See  pp.  1,  2,  9 
and  10.] 

POD  Response.  Concur.  DOD  agrees  that  confusion  exists  in 
the  definition  of  LBTS  as  contained  in  NAVMATINST  3960.8. 

The  secondary  objectives  would  not  warrant  a  LBTS  without  the 
facility  serving  its  primary  function.  For  example,  NAVSEA 
officials  thought  that  parts  of  NAVMATINST  rather  than  the 
entire  instruction  were  no  longer  valid  because  of  changes 
made  by  OPNAV  in  August  1983.  In  revising  NAVMAT  Instruction 
3960.8  to  include  software  support  activities  as  LBTS,  the 
instruction  will  address  these  activities  as  one  of  the 
possible  fuiiv.kxuua  of  a  L3TS. 

•  FINDING  D;  Complete  Inventories  Of  Existing  Navy  Land-Based 
Test  Sites  Not  Maintained.  GAO  reported  that  according  to 
the  NAVMAT  instruction,  before  a  new  test  site  is  acquired, 
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—  Integration  and  interface  testing  of  equipment  and 
subsystems ; 

—  Verification  of  subsystem  equipment; 

—  Development  or  verification  of  technical  documentation 
and  production  acceptance  test  procedures; 

—  Verification  and  validation  of  operational  and 
maintenance  computer  software; 

—  Initial  operational  or  maintenance  training; 

—  Proof  testing  alterations  or  modifications; 

—  Verifying  operator  and  crew  task  assignments; 

--  Verifying  installation  and  checkout  procedures;  and 

—  Preinstallation  testing  and  checkout  of  production 
hardware . 

GAO  noted  that  most  of  these  functions  are  also  performed  by 
other  facilities  owned  by  the  Systems  Commands  and  NAVMAT. 

GAO  found,  however,  that  the  Navy  does  not  consider  these 
other  sites  to  be  land-based  test  sites  because  they  tend  tu¬ 
be  more  general  purpose  or  functionally  oriented,  rather  1  ;  ■ 
dedicated  to  a  specific  weapon  system,  [See  pp.  6  an  i 

POD  Response.  Concur.  It  must  be  noted  that  there  is  a 
distinction  between  LBTS  and  "general  purpose  facilities". 

The  definition  in  "Finding  A"  above  for  a  LBTS  is  correct. 
LBTSs  may  and  should  be  assigned  secondary  functions  if  they 
are  to  be  cost  effective.  This  distinction  in  the  definition 
of  LBTS  in  NAVMATINST  3960.8  is  not  clear  as  written. 

•  FINDING  C;  NAVMAT  Instruction  Not  Always  Followed.  GAO 
found  that  the  NAVMAT  instruction  requirements  for 
justifying,  reviewing,  and  approving  the  acquisition  of  new 
land-based  test  sites  are  not  being  followed.  GAO  reported 
chat  there  is  confusion  among  Navy  officials  as  to  the 
applicability  of  the  NAVMAT  instruction  to  their  programs, 
while  others  were  not  even  aware  of  the  instruction. 

—  Example  1;  The  NAVMAT  China  Lake  Naval  Weapons  Center  has 
a  number  of  sites  that  provide  the  type  of  support 
authorized  by  NAVMAT  Instruction  3960.8  for  land-based 
test  sites,  but  China  Lake  officials  were  not  aware  of  the 
instruction  and  questioned  whether  it  actually  applied  to 
the  sites  they  controlled. 
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GAO  DRAFT  REPORT  -  DATED  FEBRUARY  21,  1985 
(GAO  CODE  No.  943585)  OSD  CASE  No.  6698 
“ACQUISITION  OF  NAVY  LAND-BASED  TEST  SITES  CAN  BE  BETTER  MANAGED" 
DEPARTMENT  OF  DEFENSE  COMMENTS 

***** 

FINDINGS 


•  FINDING  A:  Navy  Policy  To  Use  Land-Based  Test  Sites.  GAO 
found  that  in  the  mid-1960s  the  Navy  adopted  land-based 
testing  to  reduce  development,  construction,  and  repair 
problems.  In  the  early  1970s,  this  testing  was  expanded  to 
help  ensure  new  weapons  systems  and  their  components  were 
viable  before  acquisition  and  installation.  (Land-based  test 
sites  (LBTS)  are  facilities,  owned  either  by  the  Navy  or  its 
contractor,  which  test,  refine,  and  integrate  weapon  systems 
being  purchased  or  undergoing  changes  and  are  usually 
dedicated  to  a  specific  weapon  system.)  GAO  further  found 
that  as  weapon  systems  have  become  more  complex,  the  need  for 
land  based  test  sites  to  support  the  acquisition  of  these 
systems  has  increased.  According  to  the  Navy,  benefits 
resulting  from  land-based  test  sites  are  (1)  reduction  in  the 
amount  of  defective  contractor  material,  (2)  shorter 
construction  periods,  (3)  shorter  time  to  reach  deployable 
status,  (4)  development  of  internal  Navy  expertise,  and  (5) 
timely,  effective  life  cycle  support.  GAO  concluded  these 
land-based  test  sites  have  the  potential  to  provide 
considerable  benefits.  (See  pp\  5,  6,  14,  and  15.] 

POD  Response.  Concur. 

•  FINDING  B:  Responsibility  For  Navy  Land-Based  Test  Sites. 
GAO  found  that  organizationally,  tne  weapons  system  manager 
is  responsible  for  identifying  the  need  for  land-based  test 
sites,  with  the  review  and  approval  process  extending  up 
through  the  System  Commands,  to  the  Naval  Material  Command 

( NAVMAT)  and  then  to  the  Office  of  the  Chief  of  Naval 
Operations  (CNO) .  GAO  further  found  that  NAVMAT  is 
responsible  for  policy  and  guidance.  GAO  reported  that  the 
NAVMAT  Instruction  3960.8  governing  land-based  test  sites 
permits  a  broad  range  of  functions,  as  follows: 

—  Developmental  or  operational  test  and  evaluation; 


Enclosure  (1) 
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MANPOWER, 
INSTALLATIONS 
AND  LOGISTICS 


THE  ASSISTANT  SECRETARY  OF  DEFENSE 

WASHINGTON.  D  C  20301-4000 

4  APR  )965 


Mr.  Frank  C.  Conahar. 

Director,  National  Security  and 
International  Affairs  Division 
General  Accounting  Office 
Washington,  D.C.  20548 

Dear  Mr.  Conahan: 

This  is  in  response  to  your  letter  of  February  21,  1985  concerning  the 
draft  report  entitled.  Acquisition  of  Navy  Land-Fased  Test  Sites  Can  Be  Better 
Managed  (GAO  Code  No.  943585,  OSD  Case  No.  6698}. 

The  Department  of  Defense  generally  concurs  with  the  draft  report's 
findings  and  recommendations .  There  are,  however,  several  factual 
inaccuracies  in  the  report  which  were  brought  to  the  attention  of  your  staff 
during  a  meeting  on  March  22,  1985.  Detailed  comments  are  set  forth  in  the 
enclosure  hereto. 

The  opportunity  to  comment  on  this  report  in  draft  form  is  appreciated . 


Sincerely 


tarryc  Calhoun 


ACTG  Assistant  Secretary  of  Defense 
(Manpower,  Installations  &  Logistics) 


Enclosure 


GAO  note:  Page  references  have  been  changed  to  correspond  to 
pages  in  the  final  report. 
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— The  NAVMAT  requirement  that  systems  commands  maintain 
complete  and  up-to-date  inventories  of  in-house  and 
contractor  test  sites  be  enforced  to  aid  in  making 
acquisition  decisions,  and  the  Chief  of  Naval  Opera¬ 
tions  establish  a  central  Navy-wide  inventory  of 
test  and  evaluation  facilities. 

— Program  managers  consider  alternatives  in  selecting 
sites  and  include  their  rationale  in  the  planning 
documents. 

— The  Chief  of  Naval  Operations  and  NAVMAT  program 
review  and  approval  process  include  evaluation 
techniques  that  consider  the  cost-effectiveness  of 
sites  selected,  the  availability  of  existing  facili¬ 
ties,  and  the  extent  of  duplication  between  existing 
and  proposed  sites. 

AGENCY  COMMENTS 

On  April  4,  1985,  DOD  provided  official  written  comments  on 
a  draft  of  this  report.  (See  app.  II.)  DOD  agreed  with  each  of 
our  recommendations  and  outlined  actions  and  milestones  for 
improving  the  acquisition  management  of  land-based  test  sites. 

DOD  stated  that  NAVMAT  Instruction  3960.8  and  OPNAV  Instruction 
3960 .  1 0S  would  be  revised  by  August  15,  1985.  In  addition, 

NAVMAT  will  establish  a  central  inventory  of  land-based  test 
sites  by  December  15,  1985. 

Although  generally  agreeing  with  our  findings,  DOD 
questioned  some  of  the  details  of  our  presentation.  These 
matters  are  discussed,  where  appropriate,  in  the  individual  find¬ 
ing  sections  of  the  report. 
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This  can  lead  to  situations  where  existing  facilities  are 
underutilized  and  sites  overlap  or  duplicate  each  other.  For 
example,  in  1976  NAVSEA  established  the  Integrated  Combat  Systems 
Test  Facility,  at  a  cost  of  over  $33  million,  to  provide  software 
support  for  combat  systems  of  surface  ships.  However,  all  sur¬ 
face  ship  program  managers  have  not  been  using  the  facility  as 
anticipated  when  it  was  established.  NAVSEA* s  Aegis  program  is 
building  its  own  facilities  for  software  support  and  the  deputy 
commander  for  combat  systems  is  seeking  an  alternative  location 
for  operational  suppoft  for  the  FFG-7  program. 

In  another  case,  NAVSEA  recently  huilt  a  $7  million  guided 
missile  laboratory  at  the  Naval  Ship  Weapons  System  Engineering 
Station,  Port  Hueneme,  California.  This  laboratory,  while  not 
classified  as  a  land-based  test  site  by  NAVSEA,  oerforms  similar 
functions.  The  laboratory  was  under  construction  before  CNO's 
Test  and  Evaluation  Division  was  aware  of  its  existence,  even 
though  this  division  is  responsible  for  approving  all  Navy  test 
and  evaluation  facilities.  One  of  the  primary  i ust i f icat ions  for 
the  laboratory  was  the  support  of  NAVSEA' s  Aegis  shipbuilding 
program.  However,  Aegis  program  officials  were  unaware  that  this 
facility  was  intended  to  support  their  program.  The  program 
manager  had  selected  a  site  and  hegun  construction  of  a  similar 
laboratory  at  Wallops  Island,  Virginia. 

CONCLUSIONS  AND  RECOMMENDATIONS 

The  Navy  is  increasing  the  use  of  land-based  test  sites  to 
support  the  acquisition  of  major  weapons  systems,  ’’’hese  sites 
have  the  potential  to  provide  considerable  benefits  because  they 
can  perform  many  functions,  but  the  management  of  their  acquisi¬ 
tion  is  not  working  as  effectively  as  it  could.  Navy  instruc¬ 
tions  and  procedures  are  not  ensuring  that  sites  purchased  are 
cost-effective  and  do  not  duplicate  other  Navy  facilities. 

Navy  officials  agreed  that  problems  exist  and  have  begun  to 
make  changes  in  regulations  and  recordkeeping  activities  but 
actions  are  not  yet  complete.  While  these  actions  are  steps  in 
the  right  direction,  we  believe  more  can  be  done.  therefore,  we 
recommend  the  Secretary  of  the  Navy  direct  that: 

— The  NAVMAT  instruction  clearly  state  (1)  which 
programs  and  sites  are  subject  to  its  requirements 
and  (2)  that  the  sites  selected  be  evaluated  on  the 
basis  of  cost-effectiveness  and  needed  capability. 

--The  OPNAV  or  the  NAVMAT  instruction  be  revised  so 
that  the  instructions  agree  on  the  responsibility  for 
identifying  the  need  for  land-based  test  sites. 
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We  included  this  example  because  officials  of  the  Naval 
Ocean  Systems  rente1",  as  well  as  the  Naval  Underwater  Systems 
Center,  told  us  that  the  system  beinq  tested  was  an  unqrade  of  a 
prior  system-- not-  i  totally  new  or  different  system — and  that 
both  Centers  had  t  n-=  expertise  to  work  on  the  revised  system 
since  they  had  worked  on  prior  versions  of  the  same  system. 

Cased  on  these  discussions  and  documentation  orovided  by  NAVRF.A 
and  other  nrnan i ?at ions ,  we  do  not  believe  that  adequate 
cons  1  do r a 1 ion  was  qiven  to  existinq  sites  and  available  Naw 
expertise.  In  this  regard,  we  asked  DON  officials  to  provide  us 
the  report  resulting  from  the  1982  NAVBEA  study  and  were  told 
that  no  formal  study  report  had  been  prepared. 

As  for  the  approval  of  the  contractor  site,  the  approval  was 
made  without  a  review  by  either  NAVMAT  or  CNO  test  site  reviewinq 
officials.  ^hese  officials  said  thev  were  not  aware  of  the  olan 
to  establish  the  contractor  site  until  we  told  them.  With  reqard 
to  cost,  both  the  S50  million  figure  cited  by  us  and  the  $18 
million  figure  cited  by  DOD  were  based  on  Navy  estimates.  We  did 
not  audit  these  estimates.  Our  point  is  that  either  amount  was 
large  enough  to  warrant  cost-benefit  analyses  of  possible 
al ternat  ives . 

CNO  AND  NAVMAT  REVIEW 
PROCESS  COULD  BE  IMPROVED 


The  UNO  and  MAVMA't  review  and  approval  process  does  not 
assure  maximum  use  of  existing  facilities,  prevention  of  overlap 
and  duplication  among  sites,  or  cost-effectiveness  of  the  site 
selected.  The  lack  of  site  inventories  and  other  evaluation 
techniques  places  too  much  reliance  on  the  personal  knowledge  of 
the  reviewers. 


The  NAVMAT  review  and  approval  process  for  land-based  test 
sites  is  based  primarily  on  the  personal  knowledqe  of  the  re¬ 
viewer.  He  reviews  the  proposed  use  of  the  facility  to  determine 
if  it  will  do  tne  functions  planned  by  the  proqram  manaqer  and  is 
not  duplicative  of  existinq  facilities.  NAVMAT  officials  stated 
that  a  proposal  usually  will  be  approved  if  it  seems  loqical. 

The/  also  said  navmat  relies  on  the  personal  knowledge  and  in¬ 
tegrity  of  the  proqram  manaqer  to  have  considered  all  alterna¬ 
tives  and  to  not  request  a  site  that  is  not  needed. 

According  to  CNO  officials,  their  Test  and  Evaluation 
Division  reviews  a  proposal  in  a  similar  manner.  However,  CNO 
relies  upon  NAVMAT' s  review  and  validation  of  need  as  well  as  the 
integrity  and  knowledge  of  the  proyrar..  manaqer.  Officials  of 
both  offices  said  they  did  not  have  the  staff  expertise  available 
to  inakf1  a  detailed  evaluation  of  the  program  manager's  plan. 

They  look  for  glaring  inconsistencies  or  illogical  uses  of  the 
facility. 
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Officials  of  the  Naval  Ocean  Systems  Center  said  that  they 
had  done  essentia* 1 v  the  same  functions  for  prior  versions  of  the 
combat  system.  Accordinq  to  Center  officials,  they  had  the  Navy 
expertise  to  do  the  functions  and  could  perform  the  functions  at 
less  cost  than  the  private  contractor.  Also,  they  believed  the 
new  site  would  duplicate  the  functions  of  the  Center,  even  thouqh 
it  would  not  actually  duplicate  the  specific  work.  However,  the 
Aegis  proqram  manager  chose  to  enter  into  a  sole-source  contract 
to  establish  the  test  site  on  the  basis  that  the  qovernment  did 
not  have  the  capability  to  perform  the  work  in-house.  The  cost 
of  this  new  site,  according  to  a  Navy  estimate,  is  about  $50 
million . 

CNO  and  NAVMAT  test  site  reviewinq  officials  said  they  did 
not  know  the  site  was  beinq  purchased  because  it  had  not  been  in¬ 
cluded  in  the  planninq  documents  submitted  for  their  review. 

Aegis  program  officials  said  they  did  not  consider  this  site  a 
land-based  test  site,  and  they  did  not  follow  NAVMAT  Instruction 
3960.8.  However,  based  on  the  requirements  stated  in  the  in¬ 
struction,  we  believe  it  should  have  been  classified  as  a  land- 
based  test  site. 

In  commentinq  on  our  draft  ’report,  DOD  disagreed  that  NAVMAT 
Instruction  3960.8  does  not  make  cost-effectiveness  a  requisite 
for  establishing  a  land-based  test  site.  DOD  quoted  excerpts 
from  the  NAVMAT  instruction  to  support  its  position.  However, 

DOD  aqreed  that  cost-effectiveness  should  be  more  formally 
addressed  in  documentation. 

We  aqree  that  the  NAVMAT  instruction  infers  that 
cost-effectiveness  should  be  a  consideration  in  establishing  a 
land-based  test  site.  However,  CNO  and  NAVMAT  officials  told  us 
that  they  did  not  interpret  the  instruction  as  requirinq  an 
actual  demonstration  of  cost-effectiveness  but  rather  as  a  guide¬ 
line  for  program  manaqers  in  makinq  their  site  decisions. 
Therefore,  the  officials  did  not  require  a  cost-effectiveness 
evaluation  as  part  of  the  review  and  approval  process.  We 
believe  that  the  NAVMAT  instruction  should  provide  more  defini¬ 
tive  guidelines  to  program  manaqers  for  determining  the  cost- 
effectiveness  of  alternatives. 

DOD  stated  that  our  example  on  the  revised  antisubmarine 
warfare  combat  system  was  inaccurate  in  that  (1)  the  contractor 
site  did  not  duplicate  an  existing  land-based  test  site,  (2)  the 
functions  performed  at  the  contractor  site  were  significantly 
different  from  those  performed  at  any  Navy  activity,  (3)  costs 
associated  with  the  contractor  site  were  less  than  $15  million, 
(4)  the  Navy  approved  the  contractor  site,  and  (5)  site  selection 
documentation  was  available  in  NAVSEA.  DOD  further  stated  that 
the  decision  to  establish  the  contractor  site  resulted  from  an 
extensive  study  conducted  by  NAVSEA  in  1982.  Accordinq  to  DOD, 
this  study  considered  many  sites  and  the  contractor  site  was 
determined  to  be  the  only  one  where  the  required  activities  could 
be  accomplished  in  a  cost-effective  and  timely  manner. 
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Tne  program  officials  acquire  this  information  in  meetings 
that  are  usually  undocumented.  IE  any  documentation  is  prepared, 
it  is  generally  discarded  after  site  selection  has  been  made. 

Only  the  FFG-7  frigate  and  Aegis  cruiser  program  offices  could 
provide  documentation  of  the  rationales  used  to  select  their 
sites.  These  documents  had  been  prepared  in  response  to  congres¬ 
sional  inquiries  into  the  proposed  sites  and,  therefore,  con¬ 
tained  greater  detail  than  that  normally  included  in  planning 
documents.  Furthermore,  tnese  documents  were  prepared  more  tnan 
10  years  ago,  before  the  NAVMAT  instruction  was  issued. 

NAVMAT  officials  told  us  that  they  would  require  better 
documentation  for  proposed  sites. 

In  commenting  on  our  draft  report,  DOD  stated  that  all 
land-based  test  sites  had  approval  documentation,  but  it  was  not 
retained.  DOD  agreed  tnat  it  would  oe  useful  to  retain  records 
to  document  how  the  need  for  a  site  was  identified,  what  alterna¬ 
tives  were  considered,  and  why  the  final  alternative  was  chosen. 
DOD  also  stated  that,  in  addition  to  the  personal  knowledge  or 
expertise  of  their  staff,  program  managers  used  data  from  past 
programs,  such  as  site  management  plans  and  test  reports.  DOD 
further  stated  that  we  should  recognize  the  staff  offices  in  the 
systems  commands  that  oversee,  and  act  as  liaisons  with,  the 
various  test  and  range  facilities. 

Because  supporting  documentation  was  not  retained,  we  could 
not  verify  that  program  offices  used  data  from  prior  programs  or 
consulted  with  systems  command  staff  offices  when  selecting  a 
land-based  test  site.  We  were  able  to  review  documents  prepared 
to  gain  overall  approval  of  a  weapons  system,  such  as  decision 
papers  and  initial  test  and  evaluation  plans.  These  documents 
seldom  mentioned  a  land-based  test  site  and,  when  mentioned,  it 
generally  was  a  one  sentence  statement  that  a  site  would  be 
purchased . 

COST-EFFECTIVENESS  NOT  REQUIRED 
BY  NAVMAT  INSTRUCTION 

Tne  NAVMAT  instruction  does  not  make  cost-effectiveness  a 
requisite  for  establishing  a  new  land-based  test  site. 
Furthermore,  CNO  and  NAVMAT  officials  said  they  do  not  require 
tnat  the  program  manager  demonstrate  cost-effectiveness  before 
approving  a  site. 

NAVSEA's  Aegis  program  officials,  for  instance,  said  that  no 
cost-benefit  analyses  were  performed  when  they  decided  to 
establish  a  contractor  site  to  integrate  and  test  a  revised  anti¬ 
submarine  warfare  combat  system  for  the  DDG-51  destroyer  and  the 
Aegis  cruiser.  We  identified  an  existing  Navy  facility — 

NAVMAT* s  Naval  Ocean  Systems  Center--that  possibly  could  have 
performed  the  necessary  program  support  because  it  already  was 
iutegraL Luy  and  cescing  other  versions  of  this  combat  system. 
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